Qualitative spatial reasoning for activity recognition using tools of ambient intelligence

The aging population represents a growing concern of governments due to the extent that it will take in the coming decades and the speed of its evolution. This problem will result in increasing number of people affected by many diseases associated with aging such as the various types of dementia, including the sadly famous Alzheimer's disease. People with Alzheimer's must be assisted at all time during their everyday life. Technological assistance inside what is called a smart home could bring an affordable solution to solve this concern. One of the key issues to smart home assistance is to recognize the ongoing activities of everyday life made by the patient in order to be able to provide useful services at an appropriate moment. To do so, we must build a structured knowledge base of activities from which one or many intelligent agents (communicating with each other) would use information extracted from the various sensors to take a decision on what the inhabitant could be currently doing. The best way to build such an algorithm is to exploit constraints of different natures (logical, temporal, etc.) in order to circumscribe a library of activities. Many authors have emphasized the importance of the fundamental spatial aspect in activity recognition. However, only few works exist, and they are tested in a limited way that does not allow discerning the importance of dealing with space. Important spatial criterions, such as distance between objects, could help to reduce the number of hypotheses. Moreover, many errors can be detected only by using the spatial reasoning such as position problems (inappropriate objects are brought into the activity zone) or orientation of object issue (cup of coffee is upside down when pouring coffee). This thesis provides potential solutions to the problem outlined, which deals with spatial recognition of activities of daily living of a person with Alzheimer's disease. It proposes to adapt a theory of spatial reasoning, developed by Egenhofer, to a new model for recognition of activities. This new model allows identifying the ongoing activity using only qualitative spatial criterions which we demonstrate through the text that some could not have been identified otherwise. It also allows detection of new abnormalities related to the behavior of an individual in loss of autonomy. Finally, the model has been implemented and validated in carrying out activities in a smart home on the cutting edge of technology. These activities were derived from a clinical study with normal and mild to moderate Alzheimer subjects. The results were analyzed and compared with existing approaches to measure the contribution of this thesis. Le vieillissement de la population représente une préoccupation croissante des gouvernements en raison de l'ampleur qu'il prendra dans les prochaines décennies et la rapidité de son évolution. Ce problème se traduira par l'augmentation du nombre de personnes touchées par de nombreuses maladies liées au vieillissement telles que les différents types de démence, y compris la tristement célèbre maladie d'Alzheimer. Les personnes atteintes de la maladie d'Alzheimer doivent être assistées en tout temps dans leur vie quotidienne. L'assistance technologique à l'intérieur de ce qu'on appelle une maison intelligente pourrait apporter une solution abordable pour cette tâche. Une des questions clés inhérentes à ce type d'assistance est de reconnaître les activités courantes de la vie quotidienne faite par le patient afin d'être en mesure de fournir des services utiles au moment le plus opportun. Pour ce faire, nous devons construire une base de connaissances structurée à partir de laquelle un ou plusieurs agents intelligents utilisant l'information extraite des divers capteurs pour émettre une hypothèse ciblée concernant l'activité en cours de l'habitant. La meilleure façon de construire un tel algorithme est d'exploiter les contraintes de natures différentes (logique, temporelle, etc.) afin de circonscrire une bibliothèque d'activités. De nombreux auteurs ont souligné l'importance de l'aspect spatial fondamental dans la reconnaissance d'activité. Cependant, seuls quelques travaux existent, et ils sont testés de façon limitée qui ne permet pas de voir l'importance de considérer l'espace. Néanmoins, plusieurs critères spatiaux tels que la distance entre les objets pourraient aider à réduire le nombre d'hypothèses d'activités. Par ailleurs, de nombreuses erreurs peuvent être détectées uniquement en utilisant le raisonnement spatial, tel que les problèmes de type position ou d'orientation. Cette thèse fournit des pistes de solutions aux problèmes décrits, qui traitent de la reconnaissance spatiale des activités de la vie quotidienne d'une personne avec la maladie d'Alzheimer. Elle propose d'adapter une théorie du raisonnement spatial, développé par Egenhofer, à un nouveau modèle pour la reconnaissance des activités. Ce nouveau modèle permet d'identifier les activités en cours en utilisant uniquement les critères spatiaux. Nous démontrons à travers le texte que certaines activités ne pourraient pas avoir été identifiées autrement. Le modèle permet également la détection de nouvelles anomalies liées au comportement d'un individu en perte d'autonomie. Enfin, le modèle a été implémenté et validé en réalisant des activités dans un habitat intelligent à la fine pointe de la technologie. Ces activités ont été tirées d'une étude clinique avec des sujets normaux et Alzheimer. Les résultats ont été analysés et comparés avec les approches existantes pour évaluer la contribution de ce modèle

A new qualitative spatial recognition model based on Egenhofer topological approach using C4.5 algorithm : experiment and results

Ambient technologies and ubiquitous computing constitute together an emerging trend of research bringing new possible solutions to many problems of human life. One of them is the technological assistance of the elders suffering from cognitive deficit with their everyday life activities inside what is called a smart home. The main issue in implementing such technology is the recognition of the activities of the resident. This problem consists in inferring the minimal set of possible ongoing activities using models defined in a plans library. To achieve that, most works propose to exploit different types of constraints (logical, temporal, etc.) in order to eliminate a maximum of incoherent hypotheses. However, very few works considered exploiting the spatial aspect related to the movement of objects and to their relations in space. In this paper, we propose to add a spatial pre-filter based on a topological approach from Egenhofer to discriminate implausible ongoing activities before applying a C4.5 decision tree to choose from the remaining hypotheses. Furthermore, this paper presents promising results we obtained from an experiment on that model using real case scenarios built from clinical trials that we conducted with Alzheimer's patients

A new device to track and identify people in a multi-residents context

In recent years, technologies for monitoring people inside a house lead to the development of smart home. However, the vast majority of works deals only in monitoring the activities of a single inhabitant. Nevertheless, most of the people in the current context of ageing population does not live alone. Recognizing the activities performed by each inhabitant in a house is an important challenge. A first step to achieve this is to be able to distinguish where each inhabitant is in the house. In this paper, we present a new device to track and identify people in a multi-residents context. Experiments have been conducted to validate the reliability and accuracy of the proposed device

Method for monitoring an activity of a cognitively impaired user and device therefore

The method for monitoring an activity of a cognitively impaired user using a device having a body resting on a plurality of support areas and loadable by the cognitively impaired user, generally comprises the step of measuring a plurality of force values exerted by a weight of a load to corresponding ones of the support areas; the step of obtaining a state of an activity of the cognitively impaired user based on the measured values; and the step of generating a signal indicative of the state of the activity. The state of the activity typically can be a given step in a recipe to be cooked on a smart stove in order to assist a cognitively impaired person in the completion of a cooking activity

Spatiotemporal knowledge representation and reasoning under uncertainty for action recognition in smart homes

We apply artificial intelligence techniques to perform data analysis and activity recognition in smart homes. Sensors embedded in smart home provide primary data to reason about observations and provide appropriate assistance for residents to complete their Activities Daily Livings (ADLs). These residents may suffer from different levels of Alzheimer disease. In this paper, we introduce a qualitative approach that considers spatiotemporal specifications of activities in the Activity Recognition Agent (ARA) to do knowledge representation and reasoning about the observations. In this paper, we consider different existing uncertainties within sensors observations and Observed Agent?s activities. In the introduced approach if the more details about environment context be provided, the less activity recognition process complexity and more precise functionality is expected

Basic daily activity recognition with a data glove

Many people in the world are affected by the Alzheimer disease leading to the dysfunctionality of the hand. In one side, this symptom is not the most important of this disease and not much attention is given to this one. In the other side, the literrature provides two main solutions such as computer vision and data glove allowing to recognize hand gestures for virtual reality or robotic applications. From this finding and need, we decided to developed our own data glove prototype allowing to monitor the evolution of the dysfunctionality of the hand by recognizing objects in basic daily activities. Our approach is simple, cheap (~220$) and efficient (~100% of correct predictions) considering that we are abstracting all the theory about the gesture recognition. Also, we can access directly and easily to the raw data. Finally, the proposed prototype is described in a way that researchers can reproduce it

Unsupervised mining of activities for smart home prediction

This paper addresses the problem of learning the Activities of Daily Living (ADLs) in smart home for cognitive assistance to an occupant suffering from some type of dementia, such as Alzheimer's disease. We present an extension of the Flocking algorithm for ADL clustering analysis. The Flocking based algorithm does not require an initial number of clusters, unlike other partition algorithms such as K-means. This approach allows us to learn ADL models automatically (without human supervision) to carry out activity recognition. By simulating a set of real case scenarios, an implementation of this model was tested in our smart home laboratory, the LIARA

Mineral grains recognition using computer vision and machine learning

Identifying and counting individual mineral grainsc composing sand is an important component of many studies in environment, engineering, mineral exploration, ore processing and the foundation of geometallurgy. Typically, silt (32–128 μm) and sand (128–1000 μm) sized grains will be characterized under an optical microscope or a scanning electron microscope. In both cases, it is a tedious and costly process. Therefore, in this paper, we introduce an original computational approach in order to automate mineral grains recognition from numerical images obtained with a simple optical microscope. To the best of our knowledge, it is the first time that the current computer vision based on machine learning algorithms is tested for the automated recognition of such mineral grains. In more details, this work uses the simple linear iterative clustering segmentation to generate superpixels and many of them allow isolating sand grains, which is not possible with classical segmentation methods. Also, the approach has been tested using convolutional neural networks (CNNs). However, CNNs did not give as good results as the superpixels method. The superpixels are also exploited to extract features related to a sand grain. These image characteristics form the raw dataset. Prior to proceed with the classification, a data cleaning stage is necessary to get a usable dataset for machine learning algorithms. In addition, we present a comparison of performances of several algorithms. The overall obtained results are approximately 90% and demonstrate the concept of mineral recognition from a sample of sand grains provided by a numerical image

Activity recognition in smart homes using UWB radars

In the last decade, smart homes have transitioned from a potential solution for aging-in-place to a real set of technologies being deployed in the real-world. This technological transfer has been mostly supported by simple, commercially available sensors such as passive infrared and electromagnetic contacts. On the other hand, many teams of research claim that the sensing capabilities are still too low to offer accurate, robust health-related monitoring and services. In this paper, we investigate the possibility of using Ultra-wideband (UWB) Doppler radars for the purpose of recognizing the ongoing ADLs in smart homes. Our team found out that with simple configuration and classical features engineering, a small set of UWB radars could reasonably be used to recognize ADLs in a realistic home environment. A dataset was built from 10 persons performing 15 different ADLs in a 40 square meters apartment with movement on the other side of the wall. Random Forest was able to attain 80% accuracy with an F1-Score of 79%, and a Kappa of 77%. Those results indicate the use of Doppler radars can be a good research avenue for smart homes

Activity recognition in the city using embedded systems and anonymous sensors

This paper presents an embedded system that performs activity recognition in the city. Arduino Due boards with infrared, distance and sound sensors are used to collect data in the city and the activity, profile, and group size recognition performance of different machine learning algorithms (RF, SVM, MLP) are compared. The features were extracted based on fixed-size windows around the observations. We show that it is possible to achieve a high accuracy for binary activity recognition with simple features, and we discuss the optimization of different parameters such as the sensors collection frequency, and the storage buffer size. We highlight the challenges of activity recognition using anonymous sensors in the environment, its possible applications and advantages compared to classical smartphone and wearable based approaches, as well as the improvements that will be made in future versions of this system. This work is a first step towards real-time online activity recognition in smart cities, with the long-term goal of monitoring and offering extended assistance for semi-autonomous people